Nearest neighbor classification from multiple feature subsets

Combining multiple classiiers is an eeective technique for improving accuracy. There are many general combining algorithms, such as Bagging, Boosting, or Error Correcting Output Coding, that signiicantly improve classiiers like decision trees, rule learners, or neural networks. Unfortunately, these combining methods do not improve the nearest neighbor classiier. In this paper, we present MFS, a combining algorithm designed to improve the accuracy of the nearest neighbor (NN) classiier. MFS combines multiple NN classiiers each using only a random subset of features. The experimental results are encouraging: On 25 datasets from the UCI Repository, MFS signiicantly outperformed several standard NN variants and was competitive with boosted decision trees. In additional experiments, we show that MFS is robust to irrelevant features, and is able to reduce both bias and variance components of error.